Skip to content

feat(telemetry): add OTLP export writer#961

Open
EhabY wants to merge 4 commits into
feat/issue-903-export-telemetry-json-writerfrom
feat/issue-903-export-telemetry-otlp-writer
Open

feat(telemetry): add OTLP export writer#961
EhabY wants to merge 4 commits into
feat/issue-903-export-telemetry-json-writerfrom
feat/issue-903-export-telemetry-otlp-writer

Conversation

@EhabY
Copy link
Copy Markdown
Collaborator

@EhabY EhabY commented May 17, 2026

Summary

  • add OTLP/JSON mapping for telemetry logs, spans, and metric-shaped events
  • add .otlp.zip export writing with POST-ready logs.json, traces.json, and metrics.json
  • extend writer tests to cover signal routing and OTLP zip contents

Refs #903.

Stack: 3 / 4. Base: #960. Next: #953.

Review follow-up

  • exports spans as internal spans instead of unspecified spans
  • includes window start timestamps for HTTP percentile gauges
  • closes partially opened OTLP writers when setup fails
  • drains queued zip writes before closing file handles on error paths

Validation

  • pnpm test:extension ./test/unit/telemetry/export/range.test.ts ./test/unit/telemetry/export/files.test.ts ./test/unit/telemetry/export/writers.test.ts
  • pnpm typecheck
  • pnpm lint
  • pnpm format:check

Generated by Coder Agents.

@EhabY EhabY self-assigned this May 17, 2026
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch from 199d2e7 to 3a29ac9 Compare May 17, 2026 19:42
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from ab7dcf4 to acadd7e Compare May 17, 2026 19:42
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch from 3a29ac9 to eb776a7 Compare May 18, 2026 17:03
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from acadd7e to 29c269d Compare May 18, 2026 17:03
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch 2 times, most recently from ba095a9 to 8066586 Compare May 19, 2026 10:00
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch 2 times, most recently from 5a186aa to 5ee4862 Compare May 19, 2026 10:44
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch from 3d77d32 to c9c3b74 Compare May 19, 2026 11:02
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from 5ee4862 to 539131f Compare May 19, 2026 11:02
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch from c9c3b74 to 9c5a19d Compare May 20, 2026 10:28
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from 539131f to fe6675e Compare May 20, 2026 10:29
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch from 9c5a19d to 914025b Compare May 20, 2026 13:49
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from fe6675e to 94ac895 Compare May 20, 2026 13:50
EhabY added a commit that referenced this pull request May 21, 2026
- Move src/telemetry/export/writers.ts to writers/json.ts and the
  matching test to writers/json.test.ts. The writer keeps the same
  shape; the doc comment is tightened and the test drops a
  redundant atomic-failure assertion already covered by
  writeAtomically's own tests.
- Extract parseTelemetryTimestampMs from range.ts's
  isTimestampInRange so other export writers can reuse it.
- Add a trivial asyncIterable test helper to test/mocks/.
- Inline the duplicated `vi.mock("node:fs", ...)` /
  `vi.mock("node:fs/promises", ...)` factory pair (9 lines per file)
  as a 2-line dynamic import in the affected test files.

Upstream-compatible groundwork for the OTLP writer branch (#961);
isolates move + helper-extraction churn so that PR's diff stays
focused on OTLP-specific work.
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from 55cccf2 to 71bcce7 Compare May 21, 2026 09:55
EhabY added a commit that referenced this pull request May 21, 2026
- Move src/telemetry/export/writers.ts to writers/json.ts and the
  matching test to writers/json.test.ts. The writer keeps the same
  shape; the doc comment is tightened and the test drops a
  redundant atomic-failure assertion already covered by
  writeAtomically's own tests.
- Extract parseTelemetryTimestampMs from range.ts's
  isTimestampInRange so other export writers can reuse it.
- Add a trivial asyncIterable test helper to test/mocks/.
- Inline the duplicated `vi.mock("node:fs", ...)` /
  `vi.mock("node:fs/promises", ...)` factory pair (9 lines per file)
  as a 2-line dynamic import in the affected test files.

Upstream-compatible groundwork for the OTLP writer branch (#961);
isolates move + helper-extraction churn so that PR's diff stays
focused on OTLP-specific work.
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch from 034f5bc to 5a9edee Compare May 21, 2026 15:13
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from 7ce5f22 to d9f955a Compare May 21, 2026 15:14
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch 3 times, most recently from 1671ea2 to 5cbd8ba Compare May 21, 2026 15:51
EhabY added a commit that referenced this pull request May 21, 2026
- Move src/telemetry/export/writers.ts to writers/json.ts and the
  matching test to writers/json.test.ts. The writer keeps the same
  shape; the doc comment is tightened and the test drops a
  redundant atomic-failure assertion already covered by
  writeAtomically's own tests.
- Extract parseTelemetryTimestampMs from range.ts's
  isTimestampInRange so other export writers can reuse it.
- Add a trivial asyncIterable test helper to test/mocks/.
- Inline the duplicated `vi.mock("node:fs", ...)` /
  `vi.mock("node:fs/promises", ...)` factory pair (9 lines per file)
  as a 2-line dynamic import in the affected test files.

Upstream-compatible groundwork for the OTLP writer branch (#961);
isolates move + helper-extraction churn so that PR's diff stays
focused on OTLP-specific work.
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-json-writer branch from 5cbd8ba to d8a9b52 Compare May 21, 2026 15:54
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from d9f955a to f7308bf Compare May 21, 2026 16:00
- Restructure: writers.ts → writers/json.ts; combine OTLP mapping and
  writer into writers/otlp.ts; lift signal classification into
  export/signal.ts.
- Group all records of each signal under a single Resource block per
  file (was one Resource per event), matching the OTLP spec's
  recommended shape and shrinking exports at scale.
- Adopt OTLP enums from @opentelemetry/api + api-logs; document the
  +1 wire offset on SpanKind.
- Tighten the public surface: only writeOtlpZipExport and
  OtlpExportCounts are exported; record mappers are module-private.
- Hoist per-event metric attributes once; single-pass partition of
  http.requests measurements into counts and gauges.
- Share parseTelemetryTimestampMs between range.ts and the OTLP
  writer's nanos conversion (was duplicated with identical error).
- Extract asyncIterable test helper to test/mocks/; inline the memfs
  vi.mock pair across affected test files.
- Tests now exercise the public API only, dropping side-effect
  assertions about staging cleanup and atomic semantics already
  covered by writeAtomically's own tests.
Wire-format fixes:
- Switch http.requests count_* sums from DELTA to CUMULATIVE temporality
  (Prom/Mimir/Grafana Cloud reject the delta+sum combination).
- Maintain per-series running totals across the export, anchored at the
  first observed window's startTimeUnixNano.
- Suppress zero-cumulative counters so routes that never errored don't
  ship empty data points.

Semantic-convention compliance:
- Use OTel-standard `service.instance.id` (was `coder.session.id`) and
  `host.id` (was `coder.machine.id`) for portability with OTel-aware
  backends. The previous vendor keys were pure aliases.
- Add `schemaUrl` to each ResourceLogs/ResourceSpans/ResourceMetrics
  container and each scope container.
- Stamp scope.version with the extension version.
@EhabY EhabY force-pushed the feat/issue-903-export-telemetry-otlp-writer branch from f7308bf to 6ac4715 Compare May 21, 2026 16:14
Group OTLP code into src/telemetry/export/writers/otlp/ with one file per
concern, and hoist metric classification to src/telemetry/export/metrics.ts
so it can be shared by future writers.

Layout:
- metrics.ts:     domain (isMetricEvent, describeMetricEvent, units)
- otlp/writer.ts: public API + orchestration (signal routing, packZip)
- otlp/envelope.ts: streaming JSON envelope writer + error wrapping
- otlp/records.ts:  log/span/metric record builders + helpers
- otlp/types.ts:    OTLP wire-format interfaces (pinned to proto v1.10.0)

Safety fixes surfaced by the layer split:
- envelope: listen for stream 'error' events so failed opens reject pending
  writes instead of hanging; close() is idempotent; suffix-write failures
  during close are labeled as close failures.
- writer:   on loop failure, use Promise.allSettled to close streams without
  masking the original error.
- records:  coerce NaN/Infinity numeric inputs to 0n instead of throwing
  from BigInt(); clamp startTimeUnixNano <= timeUnixNano for out-of-order
  events; include eventName in the cumulative-series key.

Each layer has its own test file; total grows from 99 to 131 cases.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant